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This study examines the convergent validity and divergent validity of the Student Engagement in School Success 
Skills (SESSS) survey. The SESSS is easy to administer (it takes fewer than 15 minutes to complete) and is used in 
schools to provide educators with useful information about students’ use of skills and strategies related to school 
success. A total of 4,342 fifth graders completed the SESSS; the Motivated Strategies for Learning Questionnaire 
(MSLQ) Cognitive Strategy Use, Self-Regulation, Self-Efficacy and Test Anxiety subscales; and the Self-Efficacy 
for Self-Regulated Learning (SESRL). The three subscales of the SESSS (Self-Direction of Learning, Support 
of Classmates’ Learning and Self-Regulation of Arousal) correlated highly with the MSLQ Cognitive Strategy 
Use and Self-Regulation subscales, moderately correlated with the Self-Efficacy subscale and the SESRL, and 
did not correlate with the MSLQ Test Anxiety subscale. Future research is needed to use the SESSS subscales as 
discriminable dimensions. 
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For more than a decade, researchers have placed increased emphasis on evidence-based practice and a 
programmatic approach to school counseling (Carey, 2004; Green & Keys, 2001; Gysbers, 2004; Lapan, 

2005; Myrick, 2003; Paisley & Hayes, 2003; Whiston, 2002, 2011). This emphasis from the school counseling 
profession reflects national initiatives. In 2001, the Institute of Education Sciences, the research arm of the 
U.S. Department of Education, was established to determine, through rigorous and relevant research, what 
interventions are effective and ineffective for improving student achievement and education outcomes. The 
What Works Clearinghouse (WWC), an initiative of the Institute of Education Sciences, was created in 2002 to 
identify studies that provide credible and reliable evidence of the effectiveness of education interventions. The 
purpose of WWC is to inform researchers, educators and policymakers of interventions designed to improve 
student outcomes. 

The American School Counselor Association’s (ASCA, 2005) response to emerging national policy and 
initiatives included a call for school counselor-led interventions that contribute to increased student achievement 
as part of a comprehensive school counseling program. The need for more research to identify evidence-based 
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interventions tying school counselors to improved student academic performance also surfaced in a school 
counseling Delphi study, which identified the most pressing research questions in the profession (Dimmitt, 
Carey, McGannon, & Henningson, 2005). The top priority cited by this Delphi study was the need to determine 
which school counseling interventions resulted in the greatest student achievement gains. In addition, five major 
reviews of school counseling research all discussed the need for more research to strengthen the link between 
school counselor interventions and student achievement (Brown & Trusty, 2005; Dimmitt, Carey, & Hatch, 
2007; Whiston & Quinby, 2009; Whiston & Sexton, 1998; Whiston, Tai, Rahardja, & Eder, 2011). However, 
researchers continue to report limitations in the school counseling outcome research. Among the limitations are 
conclusions drawn from studies based on nonstandardized outcome assessments. For instance, in a review of 
school counseling studies, Brown and Trusty (2005) concluded that school counseling research has been limited 
by the lack of valid and reliable instruments that measure the skills, strategies and personal attributes associated 
with academic and social/relationship success. More recently, Whiston et al. (2011) completed a meta-analytic 
examination of school counseling interventions and also determined the dominance of nonstandardized 
outcome assessments in school counseling research as a significant limitation. These limitations continue to be a 
hindrance for the school counseling profession, given the goal of establishing evidence-based practices that link 
school counselor interventions to improved student outcomes. The current WWC’s Procedures and Standards 
Handbook (WWC, 2011) includes review procedures for evaluating studies that determine a particular 
intervention to be effective in improving student outcomes. The handbook provides nine reasons why a study 
under review would fail to meet WWC standards for rigorous research. Among the reasons is a failure to use 
reliable and valid outcome measures. 

While a few valid instruments have recently been developed to measure school counseling outcomes 
(Scarborough, 2005; Sink & Spencer, 2007; Whiston & Aricak, 2008), they do not measure student changes 
in knowledge and skills related to academic achievement. The Student Engagement in School Success Skills 
survey (SESSS; Carey, Brigman, Webb, Villares, & Harrington, 2013) was developed to measure student use 
of the skills and strategies that were (a) identified as most critical for long-term school success and (b) could be 
taught by school counselors within the scope of the ASCA National Model, through classroom guidance. The 
importance of continuing to evaluate the psychometric properties of the SESSS lies in the fact that, for school 
counselors, there has typically been no standardized way to measure these types of outcomes and tie them 
directly to school counselor interventions. Previous studies on self-report measures of student metacognition 
indicate that it is feasible to develop such a measure for elementary-level students (Sperling, Howard, Miller, 

& Murphy, 2002; Yildiz, Akpinar, Tatar, & Ergin, 2009). The foundational concepts, skills and strategies of 
metacognition as well as the social skills and self-management skills taught in the Student Success Skills (SSS) 
program are developmentally appropriate for grades 4-10. The questions for the SESSS were developed to 
parallel these key strategies and skills taught in the SSS program. The instrument was tested for readability and 
is appropriate for grade 4 and above. 

The SESSS is a self-report measure of students’ use of key skills and strategies that have been identified 
consistently over several decades as critically important to student success in school, as noted in large reviews 
of educational research literature (Hattie, Biggs, & Purdie, 1996; Masten & Coatsworth, 1998; Wang, Haertel, & 
Walberg, 1994a). Three skill sets have emerged from this research literature as common threads in contributing 
to student academic success and social competence: (a) cognitive and metacognitive skills such as goal setting, 
progress monitoring and memory skills; (b) social skills such as interpersonal skills, social problem solving, 
listening and teamwork skills; and (c) self-management skills such as managing attention, motivation and anger 
(Villares, Frain, Brigman, Webb, & Peluso, 2012). Additional research in support of these skills and strategies 
continues to weave a coherent research tapestry that is useful in separating successful students from students 
at risk of academic failure (Durlak, Weissberg, Dymnicki, Taylor, & Schellinger, 2011; Greenberg et al., 2003; 
Marzano, Pickering, & Pollock, 2001; Zins, Weissberg, Wang, & Walberg, 2004). 
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Linking school counseling programs and interventions to improved student outcomes has become 
increasingly important (Carey et al., 2013). One way for school counselors to demonstrate the impact 
of classroom guidance and small group counseling on achievement is by measuring the impact of their 
interventions on intermediate variables associated with achievement. These intermediate variables include the 
previously mentioned skills and strategies involving cognitive, social and self-management. Instruments that 
measure these critically important fundamental learning skills and strategies are limited. 

The present article explores the convergent and divergent validity of the SESSS (Carey et al., 2013). The 
article builds upon previous research describing the item development of the SESSS and exploratory factor 
analysis (Carey et al., 2013) and a recently completed confirmatory factor analysis (Brigman et al., 2014). The 
current findings contribute to the establishment of the SESSS as a valid instrument for measuring the impact of 
school counselor-led interventions on intermediate variables associated with improved student achievement. 

Method 

The data collected on the SESSS occurred within the context of a multiyear, large-scale, randomized control 
trial funded through the U.S. Department of Education’s Institute of Education Sciences. The purpose of the 
grant was to investigate the effectiveness of the SSS program (Brigman & Webb, 2010) with fifth graders from 
two large school districts (Webb, Brigman, Carey, & Villares, 2011). In order to guard against researcher bias, 
the authors hired data collectors to administer the SESSS, Motivated Strategies for Learning Questionnaire 
(MSLQ) and Self-Efficacy for Self-Regulated Learning (SESRL), and standardized the training and data 
collection process. The authors selected these particular surveys because they reflected factors known to be 
related to effective learning in different ways and because they provided a range of measures, some of which 
were theoretically related to the SESSS and some of which were not. 

Procedures 

During the 2011-2012 academic year, graduate students who were enrolled in master’s-level Counselor 
Education programs at two universities were hired and trained in a one-day workshop to administer the SESSS, 
MSLQ and SESRL and handle data collection materials. At the training, each data collector was assigned to five 
of the 60 schools across two school districts. After obtaining approvals from the university institutional review 
board and school district, the research team members notified parents of fifth-grade students of the study via 
district call-home systems and sent a letter home explaining the study, risks, benefits, voluntary nature of the 
study and directions on how to decline participation. One month later, data collectors entered their assigned 
schools and participating classrooms to administer the study instruments. Prior to administering the instruments, 
each data collector read aloud the student assent. Students who gave their assent were instructed to place a 
precoded generic label at the top of their instrument, and then each data collector read aloud the directions, 
along with each item and possible response choice on the SESSS, MSLQ and SESRL. Each assigned data 
collector was responsible for distributing, collecting and returning all the completed instruments to a district 
project coordinator once the data collector left the school building according to the Survey Data Collection 
Manual. In addition, the data collector noted any student absences and/or irregularities, and confirmed that all 
procedures were followed. 

The district project coordinators were responsible for verifying that all materials were returned and secured 
in a locked cabinet until they were ready to be shipped to a partner university for data analysis. The coordinators 
gathered demographic information from the district databases and matched it to the participating fifth-grade 
students and the precoded instrument labels through a generic coding system (district #1-2, school #1-30, 
classroom #1-6, student #1-25). The coordinators then saved the demographic information in a password- 
protected and encrypted Excel spreadsheet on an external device and shipped it to a partner university for data 
analysis. 
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Participants 

A total of 4,342 fifth-grade students in two large school districts completed the SESSS. The following is the 
demographic profile of the total participants: (a) gender = 2,150 (49.52%) female and 2,192 (50.48%) male; 

(b) ethnicity = 149 (3.43%) Asian, 1,502 (34.59%) Black, 865 (19.92%) Hispanic, 18 (.42%) Native American, 
125 (2.88%) Multiracial, 1,682 (38.74%) White, and 1 (.02%) no response; (c) socioeconomic status = 1,999 
(46.04%) noneconomically disadvantaged and 2,343 (53.96%) economically disadvantaged; (d) disability = 
3,677 (84.68%) nondisabled and 665 (15.32%) disabled; (e) 504 status = 4,155 (95.70%) non-504 and 187 
504 (4.3%); and (f) English language learners (ELL) = 3,999 (92.1%) non-ELL and 343 (7.9%) ELL students. 
Demographic information for fifth-grade students in each school district is reported in Table 1. 

Table 1 

Fifth-Grade Student Participant Demographics by School District 


District 1 District 2 

Demographic Characteristics (n = 2,162) (« = 2,180) 


Gender 

Female 

1,080 

(49.90%) 

1,070 

(49.10%) 


Male 

1,082 

(50.10%) 

1,110 

(50.90%) 

Ethnicity 

Asian 

89 

(04.12%) 

60 

(02.75%) 


Black 

899 

(41.58%) 

603 

(27.66%) 


Hispanic 

165 

(07.63%) 

700 

(32.11%) 


Native American 

7 

(00.32%) 

11 

(00.50%) 


Multiracial 

64 

(02.95%) 

61 

(02.80%) 


White 

938 

(43.40%) 

744 

(34.13%) 


No response 


— 

1 

(00.05%) 

SES 

Non-economically disadvantaged 

1,118 

(51.71%) 

881 

(40.41%) 


Economically disadvantaged 

1,044 

(48.29%) 

1,299 

(59.59%) 

Disability 

Nondisabled 

1,847 

(85.43%) 

1,830 

(83.94%) 


Disabled 

315 

(14.57%) 

350 

(16.06%) 

504 Status 

Non-504 

2,108 

(97.50%) 

2,047 

(93.90%) 


504 

54 

(02.50%) 

133 

(06.10%) 

English language 

Non-ELL 

1,968 

(91.03%) 

2,031 

(93.17%) 

learners 

ELL 

194 

(08.97%) 

149 

(06.83%) 


Note, n = number of students enrolled in the district; SES = socioeconomic status; ELL = English language learners. 


Instruments 

Student Engagement in School Success Skills. The SESSS (Carey et al., 2013) was developed to measure 
the extent to which students use the specific strategies that researchers have shown relate to enhanced academic 
achievement (Hattie et ah, 1996; Masten & Coatsworth, 1998; Wang et ah, 1994b). Survey items were 
written to assess students’ cognitive and metacognitive skills (e.g., goal setting, progress monitoring, memory 
skills), social skills (e.g., communication skills, social problem solving, listening, teamwork skills) and self¬ 
management skills (e.g., managing attention, motivation, anger). After the initial pool of items was developed, 
items were reviewed by an expert panel of elementary educators and school counselors and subjected to a 
readability analysis using the Lexile Framework for Reading system (MetaMetrics, 2012). At each stage of 
review, minor changes were made to improve clarity on several items. 


544 





The Professional Counselor ^olume 4, Issue 5 


Twenty-seven self-report items (plus six additional items used to control for response set) were assembled 
into a scale with the following directions: “Below is a list of things that some students do to help themselves do 
better in school. No one does all these things. No one does any of these things all the time. Please think back 
over the last two weeks and indicate how often you did each of these things in the last two weeks. Please follow 
along as each statement is read and circle the answer that indicates what you really did. Please do your best 
to be as accurate as possible. There are no right or wrong answers. We will not share your answers with your 
parents or teachers. We will not grade your answers.” 

The response format included four options that reflected frequency of strategy use in the last two weeks: “I 
didn’t do this at all,” “I did this once,” “I did this two times” and “I did this three or more times.” This response 
format was not conducive to writing clear negatively worded items; therefore, six additional items were 
developed to help control for response set. Three of these additional items reflected strategies that elementary 
students were unlikely to use (e.g., searching the Internet for additional math problems to complete). Three 
items reflected strategies that elementary students were likely to use (e.g., asking a friend when homework was 
due), but they were not covered in the SSS program. 

Based on a previous administration of the SESSS to 262 elementary students in the fourth through eighth 
grades, Carey et al. (2013) reported an overall alpha coefficient for reliability for the 27-item scale to be .91, 
and coefficient alphas for each grade ranged between .87 (for fifth grade) and .95 (for seventh grade). All items 
correlated well with the total scale (ranging between .34 and .63). Scores on the total scale were distributed 
approximately normally: M =65.83, SD = 15.44. 

In addition, Carey et al., (2013) found in an exploratory factor analysis of the SESSS scores of 402 fourth 
through sixth graders that a four-factor solution provided the best model of scale dimensionality, considering 
both the solution’s clean factor structure and the interpretability of these factors. These four factors reflected 
students’ Self-Management of Learning, Application of Learning Strategies, Support of Classmates’ Learning 
and Self-Regulation of Arousal. Regarding the SESSS factors, Self-Management of Learning and Application of 
Learning Strategies related closely to the categories of cognitive skills, metacognitive skills and the intentional 
self-regulation of cognitive processes. Support of Classmates’ Learning related closely to social skills that 
support classroom learning. Self-Regulation of Arousal related to the self-management of arousal and emotion 
that can interfere with effective learning. While determining the actual associations of SESSS factors with 
specific, previously established constructs requires empirical study, it is encouraging that the factor structure 
determined in this research corresponded with previous research. 

In a confirmatory factor analysis study (Brigman et al., 2014), using SESSS scores from a diverse sample 
of almost 4,000 fifth-grade students, who found that while a four-factor model fit the data well, the scales 
associated with Self-Management of Learning and Application of Learning Strategies correlated so highly (r 
= .90) as to be indiscriminate. These items associated with the two factors were combined, and the subsequent 
three-factor model also proved to better fit the data. Brigman et al. (2014) suggested that the SESSS is best 
thought of as having three underlying factors corresponding to Self-Direction of Learning (which represents the 
combination of the original Management of Learning and Application of Learning Strategies factors), Support 
of Classmates’ Learning and Self-Regulation of Arousal. 

Based on factor loadings, Brigman et al. (2014) created three SESSS subscales. The Self-Direction of 
Learning subscale (19 items) reflects the students’ intentional use of cognitive and metacognitive strategies to 
promote their own learning. Typical items include the following: “After I failed to reach a goal, I told myself 
to try a new strategy and not to doubt my ability,” and “I tried to keep myself motivated by imagining what 
it would be like to achieve an important goal.” The Support of Classmates’ Learning subscale (six items) 
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reflects the students’ intentional use of strategies to help classmates learn effectively. Typical items include the 
following: “I tried to help a classmate learn how to do something that was difficult for them to do,” and “I tried 
to encourage a classmate who was having a hard time doing something.” Finally, the Self-Regulation of Arousal 
subscale (three items) reflects students’ intentional use of strategies to control disabling anxiety and cope with 
stress. Typical items include the following: “I focused on slowing my breathing so I would feel less stressed,” 
and “I imagined being in a calm place in order to feel less stressed.” 

Motivated Strategies for Learning Questionnaire. The MSLQ is a 5 5 -item, student self-report 
instrument with five subscales that measure different aspects of students’ motivation, emotion, effort and 
strategy use (Pintrich & DeGroot, 1990). The different subscales of the MSLQ are designed to be used singly 
or in combination to fit the needs of a researcher (Duncan & McKeachie, 2005). Items were adapted from 
various instruments used to assess student motivation, cognitive strategy use and metacognition (e.g., Eccles, 
1983; Harter, 1981; Weinstein, Schulte, & Palmer, 1987). The present study used the following four subscales of 
the MSLQ: Cognitive Strategy Use, Self-Regulation, Self-Efficacy and Test Anxiety. 

The Cognitive Strategy Use subscale is composed of 13 items that reflect the use of different types of 
cognitive strategies (e.g., rehearsal, elaboration, organizational strategies) to support learning. Typical items 
include the following: “When I read material for class, I say the words over and over to myself to help me 
remember,” “When I study, I put important ideas into my own words” and “I outline the chapters in my book 
to help me study.” Pintrich and DeGroot (1990) reported that the Cognitive Strategy Use subscale is reliable 
(Cronbach’s alpha = .83). 

The Self-Regulation subscale includes nine items that reflect metacognitive and effort management strategies 
that support learning. Typical items include the following: “I ask myself questions to make sure I know the 
material I have been studying” and “Even when study materials are boring I keep working until I finish.” 

Pintrich and DeGroot (1990) reported that the Self-Regulation subscale is reliable (Cronbach’s alpha = .74). 

The Self-Efficacy subscale is composed of nine items that reflect students’ ratings of their level of confidence 
in their ability to do well in classroom work. Typical items include the following: “Compared with others in this 
class, I think I’m a good student” and “I know that I will be able to learn the material for this class.” Pintrich 
and DeGroot (1990) reported that the Self-Efficacy subscale is reliable (Cronbach’s alpha = .89). 

The Test Anxiety subscale includes four items that reflect students’ ratings of their experience of disabling 
levels of anxiety associated with classroom tests and examinations. Typical items include the following: “I am 
so nervous during a test that I cannot remember facts I have learned” and “I worry a great deal about tests.” 
Pintrich and DeGroot (1990) reported that the Self-Efficacy subscale is reliable (Cronbach’s alpha = .75). 

Factor analyses indicated that these MSLQ subscales are related to different latent factors. Scores on the 
Cognitive Strategy Use subscale have been demonstrated to be related to grades on quizzes and examinations, 
grades on essays and reports, and overall class grades. Scores on the Self-Regulation subscale have been shown 
to be related to the above measures plus student performance on classroom seatwork assignments. Scores on the 
Self-Efficacy subscale have been demonstrated to be related to students’ grades on quizzes and examinations, 
grades for classroom seatwork assignments, grades on essays and reports, and overall class grades. Scores on 
the Test Anxiety subscale proved to be associated with lower levels of performance on classroom examinations 
and quizzes, as well as course grades (Pintrich & DeGroot, 1990). 

Duncan and McKeachie (2005) reviewed the extensive research on the psychometric properties and research 
uses of the MSLQ. They concluded that the subscales are reliable, measure their target constructs and have been 
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successfully used in numerous studies to measure student change after educational interventions targeting these 
constructs. 

Self-Efficacy for Self-Regulated Learning scale. The SESRL was designed to measure students’ confidence 
in their abilities to perform self-regulatory strategies. It is a seven-item self-report instrument based on the 
Children’s Self-Efficacy Scale (Bandura, 2006; Usher & Pajares, 2008). Items (e.g., “How well can you 
motivate yourself to do schoolwork?”) reflect students’judgments about their abilities to perform self-regulation 
strategies identified by teachers as being frequently used by students (Pajares & Valiante, 1999). The scale has 
been used successfully with older elementary students in a self-read format and with fourth graders in a read 
aloud administration format (Usher & Pajares, 2006). Cronbach’s alpha estimates of reliability have ranged 
between .78 and .84 (Britner & Pajares, 2006; Pajares & Valiante, 2002; Usher & Pajares, 2008). Factor analysis 
has suggested that the scale is unidimensional. Concurrent validity studies have indicated that the scale is 
related to measures of self-efficacy, task orientation and achievement (Usher & Pajares, 2006). 

Data Analysis 

In the initial analysis of the three SESSS subscales, the present authors used mean imputation to replace 
missing survey responses, by replacing a missing response with the overall mean for that survey item. For each 
of the 33 SESSS items, only 8.3%-9.1% of the responses were missing. Mean imputation is appropriate when 
the percentage of missing data is less than 10% and can be considered to be missing at random (Longford, 

2005). In the current study, the students with missing survey data had an average SESSS score equal to that 
of the students with a complete response set, thus supporting the notion that the data were missing at random. 
Coefficient alpha, used as a measure of reliability, was calculated for each of the subscales before missing 
values were replaced. 

Both convergent and discriminant evidence is needed in the validation process (Campbell & Fiske, 1959; 
Messick, 1993). Messick (1993) argued that while convergent evidence is important, it can mask certain 
problems. For example, if all tests of a construct do not measure a particular facet of that construct, the tests 
could all correlate highly. Likewise, if all tests of a construct include some particular form of construct- 
irrelevant variance, then the tests may correlate even more strongly because of that fact. Due to these possible 
shortcomings of convergent evidence, discriminant evidence is needed to ensure that the test is not correlated 
with another construct that could account for the misleading convergent evidence. 

To determine the validity of the three SESSS subscales, the authors examined the correlations between 
each of the subscales with five other measures: four subscales of the MSLQ (Self-Efficacy, Cognitive Strategy 
Use, Self-Regulation and Test Anxiety), and the SESRL. Specifically, the authors considered the strength and 
direction of the SESSS subscales’ correlations with these other measures. 

Results 

Descriptive statistics and reliability estimates for the instruments used in this study are contained in Table 2. 
Coefficient alphas for the three SESSS subscales (Self-Direction of Learning, Support of Classmates’ Learning 
and Self-Regulation of Arousal), were 0.89, 0.79 and 0.68, respectively, and 0.90 for the SESSS as a whole. 
These results indicate good internal consistency (i.e., that the items within each instrument measure the same 
construct). 

All correlations between pairs of subscales appear in Table 3. Because of the large sample size in this 
study, statistical significance by itself could be misleading, so the authors used the magnitude and direction 
of the correlations for their interpretations. Correlation is an effect size reflecting the degree of association of 


547 



The Professional Counselor ^olume 4, Issue 5 


two variables (Ellis, 2010). The correlations among the three SESSS subscales ranged between .47 and .70, 
which suggests that the subscales measured related but discriminable dimensions of students’ success skill 
use. In assessing the concurrent validity of these three subscales, it was helpful to first focus on the scales 
that correlated most highly with the three SESSS subscales. The three SESSS subscales followed the same 
pattern with respect to strength of correlation. All three correlated most highly with both the Cognitive Strategy 
Use and Self-Regulation subscales of the MSLQ. These two subscales measure the students’ reported use of 
cognitive and metacognitive strategies associated with effort management and effective learning. 

Table 2 

Descriptive Statistics and Reliability Estimates for the Study Scales 


Scales 

Scales 

M 

SD 

Alpha 

SESSS 

Self-Direction of Learning 

48.6 

10.88 

0.89 


Support of Classmates’ Learning 

16.6 

4.28 

0.79 


Self-Regulation of Arousal 

7.9 

2.60 

0.68 

MSLQ 

Self-Efficacy 

28.3 

4.41 

0.83 


Cognitive Strategy Use 

48.6 

8.31 

0.82 


Self-Regulation 

31.2 

5.08 

0.75 


Test Anxiety 

8.8 

4.03 

0.79 

SESRL 

Self-Effi for Self-Reg. Learning 

31.2 

6.10 

0.83 


Note. SESSS = Student Engagement in School Success Skills; MSLQ = Motivated Strategies for Learning Questionnaire; 
SESRL = Self-Efficacy for Self-Regulated Learning. 


Table 3 

Correlations Between Scales 



SESSS 



MSLQ 





SDL SCL 

SRA 

SE 

CSU 

SR 

TA 

SESRL 

SESSS 








SDL 

0.70* 

0.58* 

0.28* 

0.54* 

0.53* 

-0.02 

0.44* 

SCL 


0.47* 

0.25* 

0.39* 

0.39* 

-0.02 

0.35* 

SRA 



0.12* 

0.32* 

0.30* 

0.08* 

0.24* 

MSLQ 








SE 




0.61* 

0.54* 

-0.38* 

0.67* 

CSU 





0.70* 

-0.20* 

0.68* 

SR 






-0.25* 

0.70* 

TA 







-0.35* 

SESRL 









Note. SESSS = Student Engagement in School Success Skills survey; MSLQ = Motivated Strategies for Learning Questionnaire; 
SESRL = Self-Efficacy for Self-Regulated Learning scale; SDL = Self-Direction of Learning; SCL = Support of Classmates’ Learning; 
SRA = Self-Regulation of Arousal; SE = Self-Efficacy; CSU = Cognitive Strategy Use; SR = Self-Regulation; TA = Test Anxiety; 

*p< .01 
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Next, the three SESSS subscales correlated with the SESRL, which measures students’ beliefs in their 
capability to engage in common effective self-regulation learning strategies. Again, all three SESSS subscales 
followed the same pattern, and had a considerable drop in magnitude of correlation with the MSLQ Self- 
Efficacy subscale, which measures students’ reports of general academic self-efficacy. While smaller, the 
correlations with the Self-Efficacy subscale (which ranged between .12 and .28) were practically different than 
0. The Self-Direction of Learning subscale had the strongest correlations with the other measures, followed by 
the Support of Classmates’ Learning subscale and finally by the Self-Regulation of Arousal subscale. There is 
evidence that the three SESSS subscales, the Cognitive Strategy Use subscale and the Self-Regulation subscale 
of the MSLQ, and the SESRL all measure some common dimension of an underlying construct, probably 
relating to students’ intentional use of strategies to promote effective learning. 

The Self-Regulation of Arousal subscale was the only one of the three SESSS subscales to be significantly 
correlated with the MSLQ Test Anxiety subscale. Conceptually, students’ abilities to self-regulate arousal 
should be related to their reported levels of test anxiety. However, given the weak relationship (r = 0.08) and the 
fact that this observed correlation was actually opposite in direction to the relationship that would be expected, 
this finding should not be given undue weight. The results of this study offer little to no evidence that the three 
SESSS subscales measure the same construct as the MSLQ Test Anxiety subscale. This weak or nonexistent 
relationship is discriminant evidence in that the other SESSS subscales did not correlate strongly or at all 
with the MSLQ Test Anxiety subscale. However, the three other MSLQ subscales and the SESRL all showed 
moderate, negative correlations with the MSLQ Test Anxiety subscale. These differing patterns of correlation 
with MSLQ Test Anxiety suggest that the SESSS subscales capture a different dimension than the common 
dimension of the underlying construct measured by the MSLQ subscales (including the Test Anxiety subscale) 
and the SESRL. 

Discussion 

The observed pattern of results is very useful in determining the validity of inferences that currently can be 
made from SESSS scores. Based on prior factor analytic studies (Brigman et al., 2014; Carey et al., 2013), the 
present authors made an attempt to create three SESSS subscales based on the items that load most strongly 
on each of three underlying factors. These three SESSS subscales showed good internal consistency and 
moderate intercorrelations suggesting that the three subscales most probably measure related but discriminable 
dimensions of students’ success skill use. However, the three SESSS subscales showed essentially the same 
pattern of correlation with comparison scales. Similarly, the pattern of results suggests that there is little or no 
overlap between the construct measured by the three SESSS subscales and the construct measured by the Test 
Anxiety subscale of the MSLQ. 

Unfortunately, these results do not shed much light on any differences among the SESSS subscales. Each 
subscale showed essentially the same pattern of correlations with the comparison scales even where differences 
would have been expected. For example, the authors would have expected the SESSS Self-Regulation of 
Arousal subscale to correlate significantly with the MSLQ Test Anxiety subscale, since individuals who are 
better able to regulate their levels of emotional arousal would be expected to experience less specific anxiety. 

These results suggest that the SESSS as a whole represents a valid measure of students’ intentional use of 
strategies to promote academic success. While prior factor analytic studies (Brigman et al., 2014; Carey et al., 
2013) have suggested that the SESSS has three related dimensions, making inferences based upon the three 
SESSS subscales related to these dimensions is not warranted. Instead, until evidence can be found that these 
three subscales measure discriminable dimensions of success skill use, the SESSS should be used as a unitary 
measure in research and practice. Future research in this area is necessary. 
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Conclusion 

Researchers have thoroughly documented the need for school counselors to demonstrate their impact on 
student achievement (Brown & Trusty, 2005; Dimmitt et ah, 2007; Whiston & Quinby, 2009; Whiston & 

Sexton, 1998; Whiston et ah, 2011). School counselor-led interventions that provide evidence of improving 
student performance remain at the top of national initiatives and research agendas (ASCA, 2005; Dimmitt et 
ah, 2005). However, there is a limited amount of standardized outcome assessments specifically tied to school 
counselor interventions available to evaluate changes in student knowledge, skills and attitudes related to 
academic achievement. 

The SESSS is easy to administer (it takes fewer than 15 minutes to complete) and educators use it in schools 
to gain valuable information about students’ use of skills and strategies related to school success. Results on the 
SESSS may be used to improve the implementation of school counselor-led interventions and reinforcement of 
specific skills in school and home settings. Current findings indicate that SESSS results should be interpreted 
as a whole rather than by subscale. SESSS results can be used to monitor student progress, and identify gaps in 
learning as well as factors affecting student behavior. 

The SESSS may be used as a screening tool to identify students in need of school counseling interventions 
and to evaluate student growth in the academic and behavioral domains. A review of SESSS student data may 
reveal gaps between student groups and identify the need for additional education opportunities, as well as lead 
to decisions about future goals of the school counseling program and discussions with administration and staff 
about program improvement (Carey et ah, 2013). Finally, SESSS student data can be used to demonstrate how 
school counselors can impact student academic and personal/social development related to classroom learning 
and achievement. SESSS results can be shared with various stakeholders through a variety of report formats 
(e.g., Web sites, handouts, newsletters), publications, or presentations at the local, regional or national level to 
document the school counselor’s ability to affect student outcomes most related to parents, administrators and 
other staff (Carey et al., 2013). 

There is one limitation in the study worth noting. While the sample size for the current analysis is considered 
large and diverse, all participates represented a single grade level, fifth grade, and two public school districts. 
Future analyses should include students from various elementary and secondary settings and grade levels. 

Future research on the psychometric properties of the SESSS should include studies that address (a) the 
reliability and intercorrelations of the assessments corresponding to the three SESSS subscales and (b) the 
predictive validity that establishes the relationships between SESSS subscales and measures of academic 
success (e.g., achievement test scores, grades, teacher ratings). These additional studies are necessary to firmly 
establish the utility of the SESSS as a reliable and valid measure of student success skills. 
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